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Background / Context: 



Elementary students from low-income families often achieve well below their middle and 
high-income peers in mathematics despite efforts in the past two decades to reform mathematics 
education (NCES, 2009; Reardon, in press). The achievement gap between children from low- 
income families and children from higher income families is present when children enter school 
and does not appear to narrow appreciably throughout schooling (Duncan & Magnuson, in 
press). Further, the gap has steadily increased over the past 25 years (Reardon, in press), raising 
the prominence of the issue in school-based efforts to improve achievement. 

Improving the quality of teaching has been raised as a promising focal point in efforts to 
improve mathematics achievement for all children, particularly children from low-income 
families (Hamre & Pianta, 2005; Jordan, Kaplan, Ola’h, and Locuniak, 2006; Nye, Konstapoulos 
& Hedges, 2004). Other school-based factors have been implicated as predictors of mathematics 
achievement in children from low income families as well. These include smaller class size (e.g. 
Krueger, 1999; Rockoff, 2003) and higher ability level of peers (Hanushek, Kain, Markman & 
Rivkin, 2003; Rockoff, 2003). However, many studies examining school-based factors on 
student achievement in mathematics have relied on value added models (e.g. Nye et al., 2004; 
Rockoff, 2003; Hanushek et al., 2003), requiring attention to what occurs inside of the classroom 
in order to guide future educational interventions. 

Classroom observational measures can provide information about high quality student- 
teacher interactions, allowing researchers to consider the impacts of these practices on student 
outcomes. Such measures can take a process-oriented approach that considers the nature of 
interactions between teachers and students, such as the sensitivity of teachers’ interactions with 
students, teachers’ effective management of the classroom, and the depth of instruction and 
quality of feedback given to students — all processes that have been linked to achievement gains 
(Pianta, Belsky, Houts & Morrison, 2007; Pianta & Hamre, 2009; Ponitz, Rimm-Kaufman, 

Brock & Nathanson, 2009). Alternatively, observational measures can take a domain-specific 
approach that considers teachers’ practices that support information processing in specific 
subject- areas, such as mathematical problem solving. Seidel and Shavelson conducted a recent 
meta-analysis that examined the effects of teaching on student learning, considering both 
domain- specific and process-oriented approaches to teaching in K-12 classrooms. They found 
that domain- specific processes had larger effects ( d = .41) on cognitive outcomes in elementary 
classrooms than all other factors (Seidel & Shavelson, 2007). The present study combines 
process-oriented and domain specific approaches, using a newly developed measure: the 
Mathematics Scan (M-Scan) Measure of Mathematics Instructional Quality (Berry, Rimm- 
Kaufman, Ottmar, Walkowiak & Merritt, 2011). 

Research Questions: 

Three primary research questions guided our analyses: 1) What is the contribution of 
mathematics instructional quality to achievement for low-income students? We hypothesized 
that mathematics instructional quality is a strong predictor of mathematics instructional quality 
for low-income students, even after controlling for prior achievement, class-size, peer ability 
level, teachers’ experience, and teachers’ content knowledge, 2) What is the relative contribution 
of classroom ability level beyond classroom quality and class size in predicting achievement for 
low-income third grade students? We hypothesized that classroom ability level would contribute 
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to achievement above and beyond mathematical instructional quality and other classroom 
factors, and 3) Does class size make a difference above and beyond instructional quality and 
classroom ability level in predicting achievement for low-income third grade students? Our 
hypothesis was that class size was significantly related to achievement above and beyond 
mathematics instructional quality and classroom ability level. 

Setting: 

Students and teachers from eleven schools in a large, suburban mid-Atlantic school 
district were studied. The schools were part of a larger randomized control efficacy study of a 
socio-emotional learning intervention, the Responsive Classroom® (RC) Approach (Northeast 
Foundation for Children, 2010). Data for the present study were collected from third grade 
students in 11 schools. These schools included students with 3-76 percent free and reduced 
lunch (M = .26), and 5-56 percent who were English-Language Learners (M = .25). 

Participants: 

All third grade children (n = 205) with designated free or reduced lunch status from 1 1 
schools were included. Students were 105 male; 108 Hispanic, 19 Caucasian, 43 African 
American, and 27 Asian American, and 169 English Language Learners. Also, 30 students 
received special education services, including 17 who were learning disabled and 9 with speech 
or language disorders. 

The students came from thirty-six different mathematics classrooms. Thirty five of the 
teachers reported demographic information, including 32 Caucasian, 3 African American, 1 
Asian and 1 Native American. Thirty-two teachers were fully licensed and 3 were provisionally 
licensed. Teachers had a mean of 11 years experience (median = 9 years, range 1-35 years). 

Intervention / Program / Practice: 

Data for the present study were collected from teachers and students at control group 
schools. These schools were studied to examine the naturally occurring variation of teaching 
quality and other classroom factors in schools without systematic intervention. 

Research Design: 

The present study examines data from control schools participating in a three-year 
longitudinal cluster randomized control trial of the RC approach. Twenty-four study schools 
were selected from the collaborating district because of their interest in training in the RC 
approach. Roughly half of selected schools were assigned randomly (with stratification for 
ethnicity and free/reduced lunch composition) to intervention and control conditions. Control 
group schools were using “business as usual” approaches to classroom management and school- 
wide social and emotional learning. 

Data Collection and Analysis: 

Baseline measures of student achievement were conducted in the spring of 2008. Teachers 
were surveyed using online questionnaires in the spring of 2009, and reported on demographic 
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information such as years’ experience and level of certification. Part of the online survey 
included an assessment of teachers ’ mathematics knowledge for teaching (MKT; Hill, Shilling, 

& Ball, 2004). Teachers received $100 for completion of the questionnaire and MKT assessment 
and for permitting classroom observations. Classroom observations were conducted and 
videotaped by research assistants in all third grade classrooms at three time points during the 
2008-2009 school year. Observations of full math lessons (approximately one hour in duration) 
occurred once in the fall (September to November), once in winter (December to February), and 
once in spring (March to May). Class size was reported by teachers during each observation. 
Finally, achievement tests in mathematics were conducted in the spring of third grade. Student 
demographic information was collected concurrently with achievement tests by school 
administrators. See Table 1 for a complete list of measures and their sources. 

Classroom observations were watched and scored by trained research assistants using the 
M-Scan (Berry et al., 2010). The M-Scan defines 8 aspects of mathematics instructional quality 
including: structure of the lesson; multiple representations; students ’ use of mathematical tools; 
cognitive depth; mathematical discourse community; explanation and justification; problem 
solving; and connections and applications. These dimensions are tied to National Council of 
Teachers of Mathematics (NCTM) Standards (2000), and capture some domain- specific 
processes that occur in mathematics classrooms such as problem-solving and the use of 
mathematical representations. To develop the M-Scan, the RCES research team relied on prior 
work by Borko and others (Borko, Stecher, Alonzo, & McClam, 2005) in the SCOOP measure, 
and Pianta and others in the CLASS measure (Pianta, LaParo, & Hamre, 2007). All coders 
attained reliability of 80% within-one of master codes for each dimension prior to beginning 
coding; monthly drift tests helped coders maintain high reliability throughout the study. Internal 
consistency for the 8 dimensions was strong, a = .93. Work establishing the validity of this 
measure is currently under review (Walkowiak, Berry, Meyer, Rimm-Kaufman & Ottmar, under 
review). 

Assumptions of normality, linearity, and homogeneity of variance were examined and 
met. Two outliers were identified and were removed. Missing data were handled using Full 
Information Maximum Likelihood approach in MPlus (Muthen, B., & Muthen, L. K., 2010). 
Analyses were conducted in 4 steps. Step one: Descriptive statistics and bivariate correlations 
were used to examine missing data, model assumptions, and associations among independent and 
dependent variables. Step two: Unconditional models were estimated using multilevel models in 
MPlus software to account for the nesting of children within classrooms (Muthen & Muthen, 
2008). Two-level models were selected in accordance with our research questions focused on 
variability in teacher quality. The intraclass correlation coefficient (ICC) was 10.4% for third 
grade achievement. Step three: Models were built incrementally, adding a level 1 control 
variable (Stanford 10 second grade mathematics score), level 2 control variables (years of 
teaching experience, mathematics knowledge for teaching ), and level 2 predictors ( classroom 
ability level, class size, and mathematics instructional quality). The level 1 and 2 variables were 
centered at the grand mean to control for their influence in the model and interpret results more 
easily (Enders & Tofighi, 2007). Step 4: Linally, to quantify the local effect size for mathematics 
instructional quality and class size, each variable of interest was added as the last predictor in the 
model, and the proportion of variance reduction (PVR) was calculated by comparing the final 
model to a model with all of the variables except for the variable of interest. 
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Results: 



Descriptive statistics and correlations for all variables can be found in Table 2. The 
results of the hierarchical linear model can be found in Table 3. The final model explains 43.9% 
of the total variance in student achievement, including 91.56% of the classroom level variance. 

RQ 1: We found that mathematical instructional quality was significantly related to third 
grade achievement for students from low-income families. For every one point higher a teacher 
scored on the M-Scan, students scored approximately 12 points higher on a third grade 
achievement test. Mathematical instructional quality explains 8% of the classroom level variance 
and .4% of the total variance in student achievement after controlling for student prior 
achievement, teachers’ content knowledge, class size, classroom ability level, and teachers’ 
experience level. 

RQ 2: Results showed the classroom ability level was not a significant predictor of 
student achievement with all of the other variables in the model. This suggests that when 
students are offered the same level of instructional quality, teacher content knowledge, teacher 
experience, and have similar class sizes, the ability level of peers is not significant. 

RQ 3 Class size was a significant predictor of achievement for students from low-income 
families. For every 3 fewer students in a classroom, students scored 1 1 points higher on the third 
grade achievement test. Class size explained 28% of the classroom level variance and 3% of the 
total variance in achievement above and beyond the other variables in the model. 

Conclusions: 

Results support prior work suggesting that high quality teachers and lower class sizes are 
both important for third grade students who come from low-income families. Specifically, 
teaching practices that are aligned with NCTM standards are related to higher achievement for 
these students. This study extends prior work from value-added models by examining the effects 
of mathematics instructional quality in addition to class size, teacher experience, mathematics 
content knowledge, and classroom ability level in predicting mathematics achievement. 

Two limitations are worth noting. First, participants in our study came from a large, well- 
funded suburban school district. In this district, administrators are able to adjust class sizes and 
personnel to classrooms where students need more help. These results may not generalize to 
samples in large urban districts that are underfunded. Second, the M-Scan measure is a recently 
developed measure. Initial findings support its validity and usefulness (Berry, Rimm-Kaufman, 
Ottmar & Merritt, 2011; Walkowiak, Berry, Rimm-Kaufman, Meyer & Ottmar, under review). 
However, results from analyses with any new measure should be interpreted cautiously. 

Although the design does not permit causal inferences, findings suggest the importance 
of mathematics instructional quality and smaller class size among students from families with 
low income. Interventions that support teacher improvement in standards-based mathematics 
may hold promise in efforts to reduce the achievement gap. For example, professional 
development for teachers that promotes high quality mathematical discourse or multiple 
approaches to problem solving may be beneficial. Future work on teacher quality should use 
observational measures that capture the nature and quality of domain- specific processes in 
classrooms, and consider whether teachers are better able to offer higher mathematics 
instructional quality when they have smaller classes. 
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Appendix B. Tables and Figures 



Table 1 

Constructs and Measures 



Construct 


Measure 


Prior Mathematics 
Achievement 


Students took the Stanford Achievement Test- Tenth Edition 
(Stanford- 10; Harcourt Educational Measurement, 2008) in the 
spring of second grade. 


Mathematics Knowledge 
for Teaching 


Teachers took the numbers and operations portion of the MKT (Hill, 
Shilling & Ball, 2004) in the spring of 2009. Scoring is based on 
national mean scores. 


Classroom Ability Level 


A classroom mean for the Stanford 10 test was computed to 
determine average ability level within each mathematics classroom. 


Class Size 


Class size data were collected during classroom observations 
throughout the third grade year. 


Mathematics 
Instructional Quality 


Research assistants rated videotaped complete math lessons at 3 
points during the year using M-Scan (Berry, Rimm-Kaufman, 
Ottmar, Walkowiak & Merritt, 2010). Each item was coded on a 1-7 
Likert scale, including low (1, 2), medium (3, 4, 5) and high (6, 7) 
quality instruction, and scores were averaged for each classroom. 


Outcome: 
Third Grade 
Mathematics 
Achievement 


Students took the Virginia Standards of Learning mathematics 
assessment in the spring of third grade (Virginia Department of 
Education, 2009). 



SREE Fall 201 1 Conference Abstract Template 



B-l 




















Table 2 

Summary of Correlations, Means and Standard Deviations for All Variables 



Measure 


1 


2 


3 


4 


5 


6 


7 


N 


203 


196 


196 


205 


205 


205 


198 


Mean 


559.06 


.02 


10.69 


567.78 


19.25 


3.37 


460.9 


Standard 

Deviation 


30.62 


.75 


8.37 


18.94 


4.00 


.81 


64.19 


Minimum 


494 


-1.16 


1 


443.93 


11 


1.75 


296 


Maximum 


662 


1.19 


35 


624.50 


31 


4.75 


600 


1. Second grade 
achievement 

2. Mathematics 


.04 














knowledge for 
teaching 
3. Years’ 




_ 27 *** 












experience 
teaching 
4. Classroom 


40*** 


.05 


29 *** 










ability level 
5. Class size 


2 i*** 


23*** 


.08 


4g*** 








6 . Mathematics 
instructional 


-.11 


.60*** 


.04 


-.15* 


-.08 






quality 

7. Third grade 

mathematics 

achievement 


oi*** 


.07 


.19* 


19*** 


-.06 


.09 





SREE Fall 201 1 Conference Abstract Template 



B-2 




Table 3 

Hierarchical Linear Model Results: Class Size and Mathematics Instructional Quality Predict 
Achievement for Third Grade Children from Low-Income Families 



Parameters 


Unconditional Model 


Final model 






Coef. 


Standard Error 


Coef. 


Standard Error 






Fixed effects 






Level 1 

Intercept (y 00 ) 


460.04*** 


4.58 


458.91*** 


4.25 


Stanford 10 (yio) 
Level 2 






1.38*** 


.15 


MKT 






.55 


5.7 


Classroom ability 






.18 


.24 


Years experience 






.45 


.54 


Mathematics 

instructional 






11.84** 


4.36 


quality 
Class size 




Random effects 


-3.66*** 


.87 


Intercept (too) 
Level 2 


429.26* 


278.32 


36.25 


113.24 


Intercept (a 2 ) 
Level 1 


3686.13*** 


409.32 


2240.79*** 


262.42 
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